Ship and ROV Core Datastreams : Nightly CTD Processing

This document describes the nightly job to load the rovctd raw data (raw logr files > raw data tables) which are then post-processed into derived data tables (~1sec and 15sec binned data) in the EXPD database. Basically, it executes a dos .bat file which in turn executes perl programs. The perl programs do all the actual work.

some history... the directory path includes folder named DTS. This is a leftover from when we initially set these loads up as DTS jobs. We eventually figured out there didn't need to be a DTS job to do this, just the ServerAgent Job. There is no DTS involved the way it is currently set up.)

This job can be found on the SQL Server 'Solstice' when logged in on an account with 'sa' privileges. This job runs as sql server user account 'sqlexec'. The files are located in a directory on mapped network drive \\Solstice\MSSQL_Data
(on Solstice this must be Drive letter E: on your local machine it can be anything)

REM
REM loadRovctd.pl contains a UNC spec to \\tornado\\shipdata
rovctd
REM
REM Must either change directory or use full path to perl scripts because
REM of how batch file gets called by DTS
REM
REM 07-Aug-2003 R.Schramm Saftey check-in to VSS. Old pre-DTS version was
REM in VSS.
REM 22-May-2008 R.Schramm modied perl script names to reflect DerFiles or DspkFiles
REM 29-May-2008 R.Schramm added call to updateRovCtdCfg.pl
REM 28-Jun-2008 R.Schramm changed to seabird ctd processing for ventana
REM

REM ppm install Statistics::Descriptive
REM ppm install DBI

REM note these commands can be issued in a dos window if you change the drive
REM letter mappings...

REM first make sure all calibrations are processed (does both vehicles)
perl E:\MSSQL\DTS\EXPD\ROVCTD\updateRovCtdCfg.pl

REM next upload raw data by vehicle
perl E:\MSSQL\DTS\EXPD\ROVCTD\loadRawRovctdLogrFiles.pl E:\MSSQL\DTS\Log\loadVentanaRawRovctdLogrFiles.log vnta
perl E:\MSSQL\DTS\EXPD\ROVCTD\loadRawRovctdLogrFiles.pl E:\MSSQL\DTS\Log\loadDocRickettsRawRovctdLogrFiles.log docr

REM next process any data successfully loaded - both vehicles
perl E:\MSSQL\DTS\EXPD\ROVCTD\processRawRovctdLogrData.pl E:\MSSQL\DTS\Log\processRawRovctdLogrData.log